Display overlays for prioritization of video subjects

ABSTRACT

Technology for generating camera viewfinder displays for camera people video recording/broadcasting live events such as sporting events, where the viewfinder displays include overlays that include: (i) priority values for objects shown on and/or off the live event view shown in viewfinder display; (ii) identifications of objects that are outside the viewfinder display; and/or (iii) direction to the locations of objects that are outside the viewfinder display. In response to these indications in the overlay, the cameraperson may move the camera to better capture a high priority object and/or capture an object that was outside the viewfinder display.

BACKGROUND

The present invention relates generally to the field of video cameras,and more particularly to displays for video cameras.

In currently conventional video cameras: (i) overlays in the viewport orfield monitor pertain to image characteristics (as opposed to real worldsubject matter shown in the image that is being displayed in the videocamera display) such as lighting; (ii) as an example of overlay data,currently conventional video cameras can overlay zebra stripes that fadeas the operator corrects for lighting; and (iii) the “tally” is a lighton the video camera that indicates whether the operating camera is live,soon to be live (preview), or neither.

SUMMARY

According to an aspect of the present invention, there is a method,computer program product and/or system that performs the followingoperations (not necessarily in the following order): (a) receiving, froma first camera device, a narrow frame video data set including dataindicative of a first view of a real world live event; (b) receiving,from a second camera device, a wide frame video data set including dataindicative of a second view of a real world live event, with the secondview including: (i) substantially all of what is visible in the firstview, and (ii) additional viewing area that is not visible in the firstview; (c) identifying, by machine logic, a set of out-of-narrow-frameobject(s) of potential interest that are visible in the additionalviewing area but not visible in the first view so that eachout-of-narrow-frame object has human understandable identification textrespectively assigned to it; (d) generating an overlay data setincluding information indicative of the identification text for eachout-of-narrow-frame object of the set of out-of-narrow-frame object(s);and (e) sending the overlay data set to the first camera device fordisplay on a display of the first camera device as an overlay overlaidon the first view.

According to a further aspect of the present invention, there is amethod, computer program product and/or system that performs thefollowing operations (not necessarily in the following order): (a)receiving, from a first camera device, a video data set including dataindicative of a first view of a real world live event; (b) identifying,by machine logic, a set of in-narrow-frame object(s) of potentialinterest that are visible in the first view so that each in-narrow-frameobject has human understandable identification text respectivelyassigned to it; (c) for each in-narrow-frame object of the set ofin-narrow-frame object(s), determining, by machine logic, a respectivepriority value; (d) generating an overlay data set including informationindicative of: (i) the identification text for each in-narrow-frameobject of the set of in-narrow-frame object(s), and (ii) the priorityvalue for each in-narrow-frame object of the set of in-narrow-frameobject(s); and (e) sending the overlay data set to the first cameradevice for display on a display of the first camera device as an overlayoverlaid on the first view.

According to a further aspect of the present invention, there is amethod, computer program product and/or system that performs thefollowing operations (not necessarily in the following order): (a)receiving, from a first camera device, a narrow frame video data setincluding data indicative of a first view of a real world live event;(b) receiving, from a second camera device, a wide frame video data setincluding data indicative of a second view of a real world live event,with the second view including: (i) substantially all of what is visiblein the first view, and (ii) additional viewing area that is not visiblein the first view; (c) identifying, by machine logic, a set ofout-of-narrow-frame object(s) of potential interest that are visible inthe additional viewing area but not visible in the first view so thateach out-of-narrow-frame object has human understandable identificationtext respectively assigned to it; (d) identifying, by machine logic, aset of in-narrow-frame object(s) of potential interest that are visiblein the first view so that each in-narrow-frame object has humanunderstandable identification text respectively assigned to it; (e) foreach in-narrow-frame object of the set of in-narrow-frame object(s),determining, by machine logic, a respective priority value; (f)generating an overlay data set including information indicative of: (i)the identification text for each in-narrow-frame object of the set ofin-narrow-frame object(s), (ii) the priority value for eachin-narrow-frame object of the set of in-narrow-frame object(s), and(iii) the identification text for each out-of-narrow-frame object of theset of out-of-narrow-frame object(s); and (g) sending the overlay dataset to the first camera device for display on a display of the firstcamera device as an overlay overlaid on the first view.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram view of a first embodiment of a systemaccording to the present invention;

FIG. 2 is a flowchart showing a first embodiment method performed, atleast in part, by the first embodiment system;

FIG. 3 is a block diagram showing a machine logic (for example,software) portion of the first embodiment system;

FIG. 4 is a first screenshot view generated by the first embodimentsystem;

FIG. 5 is a second screenshot view generated by a second embodiment of asystem according to the present invention;

FIG. 6 is a block diagram view of the second embodiment system; and

FIG. 7 is a third screenshot view generated by the third embodimentsystem.

DETAILED DESCRIPTION

Some embodiments of the present invention are directed to generatingcamera viewfinder displays for camera people videorecording/broadcasting live events such as sporting events, where theviewfinder displays include overlays that include: (i) priority valuesfor objects shown on and/or off the live event view shown in viewfinderdisplay; (ii) identifications of objects that are outside the viewfinderdisplay; and/or (iii) direction to the locations of objects that areoutside the viewfinder display. In response to these indications in theoverlay, the cameraperson may move the camera to better capture a highpriority object and/or capture an object that was outside the viewfinderdisplay.

This Detailed Description section is divided into the followingsub-sections: (i) The Hardware and Software Environment; (ii) ExampleEmbodiment; (iii) Further Comments and/or Embodiments; and (iv)Definitions.

I. The Hardware and Software Environment

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

An embodiment of a possible hardware and software environment forsoftware and/or methods according to the present invention will now bedescribed in detail with reference to the Figures. FIG. 1 is afunctional block diagram showing: networked computers system 100 andchess match 108. Networked computers system 100 includes: overlaycreation sub-system 102; narrow camera sub-system 104 (which is aprofessional digital video camera with a built in computer processor);wide camera sub-system 106 (which is a professional digital video camerawith a built in computer processor); and local area network (LAN) 114.Sub-system 102 includes: overlay creation computer 200; communicationunit 202; processor set 204; input/output (I/O) interface set 206;memory device 208; persistent storage device 210; display device 212;external device set 214; random access memory (RAM) devices 230; cachememory device 232; and program 300. Narrow camera sub-system 104includes: viewfinder 105; control module (“mod”) 107; and narrow camera109. Wide camera sub-system 106 includes: control module (“mod”) 113;and wide camera 111.

Sub-system 102 is, in many respects, representative of the variouscomputer sub-system(s) in the present invention. Accordingly, severalportions of sub-system 102 will now be discussed in the followingparagraphs.

Sub-system 102 may be a laptop computer, tablet computer, netbookcomputer, personal computer (PC), a desktop computer, a personal digitalassistant (PDA), a smart phone, or any programmable electronic devicecapable of communicating with the client sub-systems via network 114.Program 300 is a collection of machine readable instructions and/or datathat is used to create, manage and control certain software functionsthat will be discussed in detail, below, in the Example Embodimentsub-section of this Detailed Description section.

Sub-system 102 is capable of communicating with other computersub-systems via network 114. Network 114 can be, for example, a localarea network (LAN), a wide area network (WAN) such as the Internet, or acombination of the two, and can include wired, wireless, or fiber opticconnections. In general, network 114 can be any combination ofconnections and protocols that will support communications betweenserver and client sub-systems.

Sub-system 102 is shown as a block diagram with many double arrows.These double arrows (no separate reference numerals) represent acommunications fabric, which provides communications between variouscomponents of sub-system 102. This communications fabric can beimplemented with any architecture designed for passing data and/orcontrol information between processors (such as microprocessors,communications and network processors, etc.), system memory, peripheraldevices, and any other hardware components within a system. For example,the communications fabric can be implemented, at least in part, with oneor more buses.

Memory 208 and persistent storage 210 are computer-readable storagemedia. In general, memory 208 can include any suitable volatile ornon-volatile computer-readable storage media. It is further noted that,now and/or in the near future: (i) external device(s) 214 may be able tosupply, some or all, memory for sub-system 102; and/or (ii) devicesexternal to sub-system 102 may be able to provide memory for sub-system102.

Program 300 is stored in persistent storage 210 for access and/orexecution by one or more of the respective computer processors 204,usually through one or more memories of memory 208. Persistent storage210: (i) is at least more persistent than a signal in transit; (ii)stores the program (including its soft logic and/or data), on a tangiblemedium (such as magnetic or optical domains); and (iii) is substantiallyless persistent than permanent storage. Alternatively, data storage maybe more persistent and/or permanent than the type of storage provided bypersistent storage 210.

Program 300 may include both machine readable and performableinstructions and/or substantive data (that is, the type of data storedin a database). In this particular embodiment, persistent storage 210includes a magnetic hard disk drive. To name some possible variations,persistent storage 210 may include a solid state hard drive, asemiconductor storage device, read-only memory (ROM), erasableprogrammable read-only memory (EPROM), flash memory, or any othercomputer-readable storage media that is capable of storing programinstructions or digital information.

The media used by persistent storage 210 may also be removable. Forexample, a removable hard drive may be used for persistent storage 210.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer-readable storage medium that is also part of persistent storage210.

Communications unit 202, in these examples, provides for communicationswith other data processing systems or devices external to sub-system102. In these examples, communications unit 202 includes one or morenetwork interface cards. Communications unit 202 may providecommunications through the use of either or both physical and wirelesscommunications links. Any software modules discussed herein may bedownloaded to a persistent storage device (such as persistent storagedevice 210) through a communications unit (such as communications unit202).

I/O interface set 206 allows for input and output of data with otherdevices that may be connected locally in data communication with servercomputer 200. For example, I/O interface set 206 provides a connectionto external device set 214. External device set 214 will typicallyinclude devices such as a keyboard, keypad, a touch screen, and/or someother suitable input device. External device set 214 can also includeportable computer-readable storage media such as, for example, thumbdrives, portable optical or magnetic disks, and memory cards. Softwareand data used to practice embodiments of the present invention, forexample, program 300, can be stored on such portable computer-readablestorage media. In these embodiments the relevant software may (or maynot) be loaded, in whole or in part, onto persistent storage device 210via I/O interface set 206. I/O interface set 206 also connects in datacommunication with display device 212.

Display device 212 provides a mechanism to display data to a user andmay be, for example, a computer monitor or a smart phone display screen.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

II. Example Embodiment

FIG. 2 shows flowchart 250 depicting a method according to the presentinvention. FIG. 3 shows program 300 for performing at least some of themethod operations of flowchart 250. This method and associated softwarewill now be discussed, over the course of the following paragraphs, withextensive reference to FIG. 2 (for the method operation blocks) and FIG.3 (for the software blocks).

Processing begins at operation S255, where narrow camera sub-system 104and wide camera sub-system send their respective video signals (made upof video frames) to receive videos mod 302 of program 300 of overlaycreation sub-system 102 (see FIG. 1). In this example, the live evenbeing capture, simultaneously, by the two video cameras, is a chessmatch between Abel and Baker. As shown in FIG. 1: (i) narrow camera 109of narrow camera sub-system 104 is configured to capture a relativelynarrow frame of view (which narrowness of frame of view is shown inviewfinder screen shot 402 of FIG. 4); and (ii) wide camera 111 of widecamera sub-system 106 is configured to capture a relatively wide frameof view.

In this example, the narrow and wide cameras have separate and distinctcamera points and focal points that are widely spaced apart. Thisexample was chosen that way largely to help make the drawings easier tounderstand and the basic concepts. Alternatively, in other embodiments,wide and narrow cameras may share much of their hardware and have focalpoints that are close to each other, or even identical. For example, thesame image capturing hardware could be used in the wide camera and thenarrow camera, with the wide and narrow cameras only being distinguishedby the fact that the narrow camera crops a larger image that is theimage of the wide camera. In the parlance of this document, that isconsidered as two cameras, even though a layperson would typically,colloquially refer to this as a single camera that simultaneouslyproduces two images.

Processing proceeds to operation, S260, where identify objects mod 304identifies all of the objects of potential interest (for example,people) within the video data received from the narrow and wide camerasat operation S255. In this example, and as shown in FIGS. 1 and 4, theseobjects of potential interest are: chess player Abel (shown only in thewide shot); chess player Baker; a chess clock and a chess board with anactive chess match being played upon it. The next sub-section of thisDetailed Description section has further Details about how computers canbe used to identify objects of potential interest that are visible inthe video of video signal(s).

Processing proceeds to operation S265, where calculate interest valuesmod 306 assigns viewer interest values to objects in the narrow framesreceived from the narrow camera. “Interest values” are one type of“priority values.” “Priority values” are any values that rank objectsbased upon how important they are to a cameraperson covering a liveevent. Different types of live events will typically have differenttypes of factors that feed into the calculation of a priority value. Inthis example, the interest values are limited to considering howinteresting a given object is to a chess viewer. For example, the chessboard is considered as interesting to chess enthusiast viewers. However,imagine a cameraperson covering the live event that is a piano concertoin a public park—in that example, the presence of a chessboard in thepark would have relatively little interest to the typical concertoviewer. Even beyond expected viewer interest, other factors may feedinto the calculation of priority values, such as the required coverageof corporate logos discussed in the following sub-section of thisDetailed Description section.

In this example, only objects visible in the narrow frame are assignedpriority values. Alternatively, all objects of interest in both thenarrow and wide frames may be assigned priority values.

In this example, two different objects may be given the same priorityranking. Alternatively, priority rankings may be calculated so that notwo objects are assigned equal priority values.

Processing proceeds to operation S270, where create overlay mod 308assembles an overlay data set to provide information indicative of: (i)objects of interest that are just offscreen (that is, in the wide frame,but not yet in the narrow frame); and (ii) priority values determined atoperation S265. Alternatively, some embodiments of the present inventionmay provide only one of these two types of overlay information. As afurther alternative, some embodiments of the present invention mayadditionally provide for other types of overlay information.

Processing proceeds to operation S275, where send out overlay mod 310sends the overlay data set to control mod 107 of narrow camerasub-system 104 through LAN 114 (see FIG. 1).

Processing proceeds to operation S280, where screen shot 402 of chessmatch 108 is shown on viewfinder 105 (see FIGS. 1 and 4). As can readilybe seen in screen shot 402: (i) the chess board and chess clock bothhave a priority value of A (highest priority); (ii) Baker's face has apriority value of B; and (ii) Abel is offscreen to the left of thenarrow frame. This information helps the cameraperson which alternativestrategy to use to proceed to the next shot. In this example, suchstrategies include: (i) panning the camera left to see Abel's face; (ii)zooming in to a close up of the chess board; (iii) zooming in to a closeup of the chess clock; (iv) doing a medium shot of the chess board andclock; or (v) zooming in to a close up of Baker's face.

Processing proceeds to operation S285, where the camera operatorresponds to the overlay data shown on her viewfinder in operation S280.More specifically, in this example, the cameraperson (who has moreexperience shooting tennis matches than chess matches) had been planningto shift to a close up of Baker's face, but the machine logic of program300 has determined that the board and clock would be more interesting tochess enthusiast viewers, so the cameraperson instead opts for a mediumshot that captures both the chess board and the chess clock, but not thefaces of either player. Although not shown in flowchart 250, at thispoint processing may loop back to operation S255 as the chess matchcontinues and develops.

III. Further Comments and/or Embodiments

Some embodiments of the present invention recognize one, or more, of thefollowing facts, potential problems and/or potential areas forimprovement with respect to the current state of the art: (i) liveevents with video coverage seek to provide their audience with the bestcontent while also highlighting their sponsors; (ii) for high profileevents, a technical director is responsible for meeting these goals;(iii) the technical director's role is to instruct the cameraoperator(s) on what to record; (iv) the technical director mustconstantly provide clear direction to each camera operator to ensurethat the correct subjects are being filmed at just the right time; (v)the difficulty of this job is magnified when there are many cameraoperators to direct; (vi) effective direction requires tremendousknowledge of the subject matter in addition to direction skill; (vii)broadcasters have different personalities and interests; (viii)depending on the broadcaster personality, a person might have anaffinity towards a certain action; (ix) the generated text from videothat is translated into slang and personal styles that each broadcasterwould say; and/or (x) as a result, the director can get an idea of whatthe broadcasters will say with a particular scene.

Some embodiments of the present invention may include one, or more, ofthe following features, characteristics and/or advantages: (i) head-updisplay overlays within video camera viewport for prioritization ofvideo subjects using digital analytics and character based naturallanguage generation; (ii) employs visual recognition technology andvideo data sources to provide head-up overlays onto the electronicviewfinder and field monitor of a video camera; (iii) the video cameraoperator is presented with this information as she is filming, so shecan focus on high value video subjects; (iv) provides image to textnatural language generation so the video the video producer can gainideas about what broadcasters will say before panning cameras ordirecting a cut; (v) creates automatic text that is translated to matchthe personality of the broadcasters; (vi) identifies the objects in thecamera operator's view and presents these labels in the display; (vii)subject labels are queried against data sources, such as web analyticsand social media analytics, to provide prioritization; and/or (viii)utilizes a secondary, ultra-wide angle camera on the camera operator'sdevice specifically for the purpose of identifying subjects andproviding cues to the operator.

Some embodiments of the present invention may include one, or more, ofthe following features, characteristics and/or advantages: (i) provides“automated technical direction” because the overlays on the cameraoperator's view provide direction; (ii) this assistance simplifies boththe job of technical director and the job of camera operator; (iii) theprovided data also lessens the need for deep subject matter expertise,so skilled directors may work across domains; and/or (iv) the overallproduction is enhanced by ensuring that high value video subjects arecaptured at the correct time.

Some embodiments of the present invention may include one, or more, ofthe following features, characteristics and/or advantages: (i) secondarycamera on camera operator's device captures video footage to be analyzedby video recognition software; (ii) video recognition softwareidentifies video subjects; (iii) identified logos are prioritizedaccording to pre-determined goals for frequency; (iv) identified peopleare prioritized according to relative popularity on owned digitalplatforms and/or social media; (v) video subject labels andprioritization are provided in the head-up display, overlaid onto theimage in the electronic viewfinder and field monitor; and/or (vi)convert image scenes into natural text while translating the text intoindigenous broadcaster language.

Some embodiments of the present invention may include one, or more, ofthe following features, characteristics and/or advantages: (i) use webanalytics and/or social media analytics to prioritize video subjects andpresenting this information in a head-up display within the cameraoperator's view; (ii) use a second camera for the detection of highvalue subjects and for presenting cues via the head-up display; (iii)translation of scene points into broadcaster character specific domainlanguage; and/or (iv) sequential scene epicenter stitching based onautomatic specific domain language, excitement scores and visual cues.

In connection with the head-up display overlays within video cameraviewport for prioritization of video subjects using digital analyticsaccording to some embodiments of the present invention, the term“augmented reality” describes embodiments where the camera operator'sexperience of the real world through the viewfinder is augmented bydigital data overlays.

In some embodiments of the present invention, the system components mayinclude the following: (i) camera operator, that is person who operatesthe video camera; (ii) technical director, that is the person whoinstructs the camera operator(s) on what to record; (iii) main camera,that is the camera recording footage for the video production; (iv)secondary camera, that is, ultra-wide angle camera used for theidentification of video subjects; (v) video subjects, that is potentialitems to be filmed in the video production, may include people, objectsand/or brand logos; (vi) head-up display, that is, overlays providinglabels, prioritization and/or data concerning the video subjects; (vii)visual recognition technology software, that is software trained toidentify video subjects within video footage; and (viii) web analytics,that is data that comes from website usage, which can be useful forprioritizing the audience's interest in various subjects.

With respect to item (vi) on the list in the preceding paragraph, thehead-up display is also sometimes referred to as a “viewfinder/fieldmonitor head-up display,” as an example of a viewfinder/field monitorhead-up display is shown in screen shot 500 of FIG. 5. In this example,an embodiment of the present invention provides three AR captions: (i)P1: Joe Golfer (indicating that the person underneath the caption hasbeen recognized by machine logic as a person named “Joe Golfer” and thathe is the object in the screenshot with the highest priority level;(ii)<<15 degrees: Company Logo (indicating that the object having thesecond highest priority is a Company Logo, off screen by 15 degrees tothe left of the current camera orientation); and (iii) P3: MashieNiblick 10 degrees>>(indicating that the object having the third highestpriority is another golfer named Mashie Niblick, off screen by 10degrees to the right of the current camera orientation).

According to an embodiment of the present invention, process flowincludes the following: (i) video recognition technology is trained toidentify relevant video subjects such as athletes, celebrities and/orbrand logos; (ii) technical director inputs desired frequency forshowing brand logos in video coverage (for example, show soft drinkbrand signage at least every 10 minutes); (iii) video camera's maincamera is recording live video footage that is used in the videoproduction; (iv) secondary camera (ultra-wide angle) is recording livevideo solely for video subject identification; (v) video footage fromsecondary camera is analyzed using visual recognition technology,identifying the people, objects, and sponsored brand logos present inthe frame; (vi) the identified items are queried against data sources toassess their priority as video subjects; (vii) a head-up displaypresents overlays in the camera's electronic viewfinder and fieldmonitor. The overlays denote the identified people and logos and displaypriority for each; (viii) when the secondary camera identifies highpriority subjects outside of the main camera's view, cues indicate thepresence of the items and their location relative to the main cameraview; and (ix) the camera operator will reposition or refocus the maincamera view in order to capture the highest priority subjects. Thisprocess flow is shown in block diagram 600 of FIG. 6.

Three (3) potential use cases of various embodiments of the presentinvention will respectively be discussed in the following threeparagraphs.

FIRST USE CASE: camera operator is filming golf but is not a subjectmatter expert. Operation of an embodiment of the present inventionassists her by providing a label indicating the golfer who has been“favorited” by many users on the tournament's mobile application.

SECOND USE CASE: a relatively unknown player is quickly becoming verypopular due to a viral video posted on social media. When an embodimentof the present invention queries his name against social mediaanalytics, the high volume of social conversation will cause theplayer's priority to be increased.

THIRD USE CASE: the production aims to feature a predetermined softdrink brand logo signage every 10 minutes during the broadcast. Thesystem records the timestamp each time the logo is identified within thefootage. After 10 minutes have elapsed without a recognition of thelogo, the logo becomes the video subject with the top prioritization.The secondary camera recognizes the location of the logo just outsidethe of the main camera's view. Then, the heads-up display indicates thatthe logo can be in frame if the operator turns 10 degrees to her right.

FIG. 7 shows an example screen shot 700 according to an embodiment ofthe present invention.

The machine logic of the present invention may assign prioritization toobjects in, or near, the camera display according to different factorsdepending upon the context of the event being videoed. For example, inthe context of a sporting event, the prioritization typically reflectswhich video subjects the broadcast values the most at that moment. Theprioritization is informed by sponsorship considerations, the expectedexcitement level of the subject (excitement vector) and digitalanalytics data (popularity on web, mobile, social). The primary consumerof the prioritization is the camera operator because she will want tofocus on the subjects that are the highest priority for the broadcast.The secondary consumer is the technical director who decides what camerafeeds are used in the broadcast. The prioritization helps the directormake her decision on which feed to select.

The term “web analytics” is used herein to refer to data that iscollected from the broadcasting organization's owned digital properties(website, mobile app, TV app, smartwatch app, etc.) Web analytics datais collected by the digital property as users interact with theproperty. Web analytics are particularly useful in determining featureusage, geolocation, and device characteristics. These elements can alsobe correlated with one another to discover subsets of users who sharecharacteristics and/or usage patterns. For example, a relevant use ofweb analytics for a golf tournament live broadcast could be to query thenumber of page views for various player profiles to infer which playersthe audience prefers and, therefore, who should be prioritized. Thiscase can be extended by querying only the profile views generated byusers located in the specific country for which the video stream isintended. Commonly used web analytics tools include IBM DigitalAnalytics, Adobe Analytics (AKA Omniture, SiteCatalyst), GoogleAnalytics.

The term “social media analytics” is used herein to refer to two (2)distinct types of social media analytics data that various embodimentsof the invention can use: (i) data that pertains to platform-wide socialmedia usage around a particular subject (this is also called “socialmedia listening,” and it is useful when an organization wants to respondto a current trend (for example, elevating the prioritization of JoeGolfer as mentions of “Joe Golfer” are increasing platform-wide on asocial media site, like Twitter)); and/or (ii) data that pertains onlyto activity related to the broadcasting organization's owned account.With respect to item (ii) of the foregoing list, this feedback helpsorganizations determine what type of content is well-received by theiraudience (for example, elevating the prioritization of Joe Golfer due toa high number of user engagements with a specific photo featuring JoeGolfer that was posted using an official Twitter account belonging to agolf tournament in which Joe Golfer participates. Commonly used socialmedia platforms include Facebook, Twitter, Instagram, Snapchat, andPinterest.

Currently, video cameras do not consider the relative value among thesubjects being recorded. Our invention is a technological advancementbecause it determines these relative values using digital analytics andmakes this information available to camera operators through the head-updisplay. Further, the introduction of a secondary camera dedicated toidentifying high value subjects is a technological advancement. Lastly,generating broadcaster-specific commentary enables the technicaldirector to select the best feeds for the broadcast.

Some embodiments of the present invention may improve computer and videocamera capabilities in one, or more, of the following ways: (i) usingweb analytics and/or social media analytics to prioritize video subjectsand presenting this information in a head-up display within the cameraoperator's viewport; (ii) the use of a second camera for the detectionof high value subjects and for presenting cues via the head-up display;(iii) translation of scene points into broadcaster character specificdomain language; and/or (iv) sequential scene epicenter stitching basedon automatic specific domain language, excitement scores and visualcues.

Some embodiments of the present invention are directed to sequentialscene epicenter stitching based on automatic specific domain language,excitement scores and visual cues. Live events with video coverage seekto provide their audience with the best content while also highlightingtheir sponsors. For high profile events, a technical director isresponsible for meeting these goals. The technical director's role is toinstruct the camera operator(s) on what to record. The technicaldirector must constantly provide clear direction to each camera operatorto ensure that the correct subjects are being filmed at just the righttime. Some embodiments of the invention can be seen as “automatedtechnical direction” because the overlays on the camera operator's viewprovide direction regarding sequential epicenter stitching. Thisassistance simplifies both the job of technical director and the job ofcamera operator. The provided data also lessens the need for deepsubject matter expertise, so skilled directors may work across domains.Most importantly, the overall production is enhanced by ensuring thathigh value video subjects are captured at the correct time.

IV. Definitions

Present invention: should not be taken as an absolute indication thatthe subject matter described by the term “present invention” is coveredby either the claims as they are filed, or by the claims that mayeventually issue after patent prosecution; while the term “presentinvention” is used to help the reader to get a general feel for whichdisclosures herein are believed to potentially be new, thisunderstanding, as indicated by use of the term “present invention,” istentative and provisional and subject to change over the course ofpatent prosecution as relevant information is developed and as theclaims are potentially amended.

Embodiment: see definition of “present invention” above—similar cautionsapply to the term “embodiment.”

and/or: inclusive or; for example, A, B “and/or” C means that at leastone of A or B or C is true and applicable.

Including/include/includes: unless otherwise explicitly noted, means“including but not necessarily limited to.”

Module/Sub-Module: any set of hardware, firmware and/or software thatoperatively works to do some kind of function, without regard to whetherthe module is: (i) in a single local proximity; (ii) distributed over awide area; (iii) in a single proximity within a larger piece of softwarecode; (iv) located within a single piece of software code; (v) locatedin a single storage device, memory or medium; (vi) mechanicallyconnected; (vii) electrically connected; and/or (viii) connected in datacommunication.

Computer: any device with significant data processing and/or machinereadable instruction reading capabilities including, but not limited to:desktop computers, mainframe computers, laptop computers,field-programmable gate array (FPGA) based devices, smart phones,personal digital assistants (PDAs), body-mounted or inserted computers,embedded device style computers, application-specific integrated circuit(ASIC) based devices.

What is claimed is:
 1. A computer-implemented method (CIM) comprising:receiving, from a first camera device, a narrow frame video data setincluding data indicative of a first view of a real world live event;receiving, from a second camera device, a wide frame video data setincluding data indicative of a second view of a real world live event,with the second view including: (i) substantially all of what is visiblein the first view, and (ii) additional viewing area that is not visiblein the first view; identifying, by machine logic, a set ofout-of-narrow-frame object(s) of potential interest that are visible inthe additional viewing area but not visible in the first view so thateach out-of-narrow-frame object has human understandable identificationtext respectively assigned to it; generating an overlay data setincluding information indicative of the identification text for eachout-of-narrow-frame object of the set of out-of-narrow-frame object(s);and sending the overlay data set to the first camera device for displayon a display of the first camera device as an overlay overlaid on thefirst view.
 2. The CIM of claim 1 wherein: the narrow frame video dataset is received from the first camera device over a communicationnetwork; the wide frame video data set is received from the secondcamera device over the communication network; and the overlay data setis sent to the first camera device over the communication network. 3.The CIM of claim 1 wherein the first camera device and the second cameradevice are housed in a common camera housing.
 4. The CIM of claim 1wherein the first camera device and the second camera device include anduse common image capturing hardware.
 5. The CIM of claim 1 furthercomprising: for each given out-of-narrow-frame object of the set ofout-of-narrow-frame object(s), determining, by machine logic, adirection between the given out-of-narrow-frame object and a closestpoint to the given out-of-narrow-frame object and an edge of the firstview.
 6. The CIM of claim 5 wherein the generation of the overlay dataset includes: for each given out-of-narrow-frame object of the set ofout-of-narrow-frame object(s), placing the information indicative of theidentification text for the given out-of-narrow-frame object in avicinity of the closest point on the edge of the first view to the givenout-of-narrow-frame object.
 7. The CIM of claim 5 wherein the generationof the overlay data set includes: for each given out-of-narrow-frameobject of the set of out-of-narrow-frame object(s), including in theoverlay data set human understandable information indicative of thedirection from the closest point on the edge of the first view to thegiven out-of-narrow-frame object.
 8. The CIM of claim 1 wherein the setof out-of-narrow-frame object(s) includes a corporate logo.
 9. The CIMof claim 1 wherein the real world live event is a sporting event. 10.The CIM of claim 9 wherein the set of out-of-narrow-frame object(s)includes a player participating in the sporting event.
 11. The CIM ofclaim 1 further comprising: displaying, on the first camera device, theoverlay data set as an overlay overlaid on the first view.
 12. The CIMof claim 11 further comprising: responsive to the display of the overlaydata set, moving the first camera so that a first out-of-narrow-frameobject appears in the first view.
 13. A computer-implemented method(CIM) comprising: receiving, from a first camera device, a video dataset including data indicative of a first view of a real world liveevent; identifying, by machine logic, a set of in-narrow-frame object(s)of potential interest that are visible in the first view so that eachin-narrow-frame object has human understandable identification textrespectively assigned to it; for each in-narrow-frame object of the setof in-narrow-frame object(s), determining, by machine logic, arespective priority value; generating an overlay data set includinginformation indicative of: (i) the identification text for eachin-narrow-frame object of the set of in-narrow-frame object(s), and (ii)the priority value for each in-narrow-frame object of the set ofin-narrow-frame object(s); and sending the overlay data set to the firstcamera device for display on a display of the first camera device as anoverlay overlaid on the first view.
 14. The CIM of claim 13 whereindetermination of the priority value(s) is based, at least in part, uponrelative expected viewer interest in the object(s) of thein-narrow-frame-object(s).
 15. The CIM of claim 13 wherein determinationof the priority value(s) is based, at least in part, upon contractualobligations to display certain objects.
 16. The CIM of claim 13 wherein:the real world live event is a sporting event; a first in-narrow-frameobject of the set of in-narrow-frame object(s) is a player participatingin the sporting event; and determination of the priority value(s) forthe first in-narrow-frame object is based, at least in part, upon anathletic posture of the player.
 17. The CIM of claim 13 wherein: a firstin-narrow-frame object of the set of in-narrow-frame object(s) is aperson; and determination of the priority value(s) for the firstin-narrow-frame object is based, at least in part, upon emotiondisplayed by the person.
 18. The CIM of claim 13 further comprising:displaying, on the first camera device, the overlay data set as anoverlay overlaid on the first view.
 19. The CIM of claim 18 furthercomprising: responsive to the display of the overlay data set, movingthe first camera based upon the priority value(s).
 20. Acomputer-implemented method (CIM) comprising: receiving, from a firstcamera device, a narrow frame video data set including data indicativeof a first view of a real world live event; receiving, from a secondcamera device, a wide frame video data set including data indicative ofa second view of a real world live event, with the second viewincluding: (i) substantially all of what is visible in the first view,and (ii) additional viewing area that is not visible in the first view;identifying, by machine logic, a set of out-of-narrow-frame object(s) ofpotential interest that are visible in the additional viewing area butnot visible in the first view so that each out-of-narrow-frame objecthas human understandable identification text respectively assigned toit; identifying, by machine logic, a set of in-narrow-frame object(s) ofpotential interest that are visible in the first view so that eachin-narrow-frame object has human understandable identification textrespectively assigned to it; for each in-narrow-frame object of the setof in-narrow-frame object(s), determining, by machine logic, arespective priority value; generating an overlay data set includinginformation indicative of: (i) the identification text for eachin-narrow-frame object of the set of in-narrow-frame object(s), (ii) thepriority value for each in-narrow-frame object of the set ofin-narrow-frame object(s), and (iii) the identification text for eachout-of-narrow-frame object of the set of out-of-narrow-frame object(s);and sending the overlay data set to the first camera device for displayon a display of the first camera device as an overlay overlaid on thefirst view.